Requested Patent: EP0713335A2 



Title: 

SYSTEM AND METHOD FOR WIRELESS CAPTURE OF ENCODED DATA 
TRANSMITTED WITH A TELEVISION. VIDEO OR AUDIO SIGNAL AND 
SUBSEQUENT INITIATION OF A TRANSACTION USING SUCH DATA ; 

Abstracted Patent: EP071 3335 ; 

Publication Date: 1996-05-22 ; 

lnventor(s): 

SIZER II THEODORE (US); HERBST STEVEN M (US); AUGUST KATHERINE G 
(US); CALDWELL CHARLES D (US) ; 

Applicant(s): AT .T CORP (US) ; 

Application Number EP1 9950307885 1 9951 1 06 ; 

Priority Number(s): US1 9940339538 19941115 ; 

IPC Classification: H04N7/08 ; 

Equivalents: CA2162614, JP8289042 

ABSTRACT: 

A device (110) which can be portable and therefore used in conjunction wXh several different 
date sources is arranged for wireless capture of encoded non-perceptible data direcUy from a 
light emitting device, such as a television (1 01 ) or other video and/or audio output device. The 
data is part of the ordinary program displayed by the device (101) and yet indiscernible by a 
viewer or listener. The indiscernible data may be information needed to order those services or 
products, including prices, delivery intervals, shipping details, coupon offers, and so-on. After 
capture, the device includes a memory arranged to store the data for later use, and 
communications circuitry (114 or 1 1 8) arranged to output at least portions of the data (or data 
derived from the captured data) to a remote agent or device (120,121 ), such as via a telephone 
call (via 140 or 1 09 or 1 12) or a data transfer to a point of sale device. 
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(54) System and method forwirelesscaptureof encoded data transmitted with atelevlslon, video 
or audio signal and subsequent initiation of a transaction using such data 



(57) A device (110) which can be portable and there- 
fore used in conjunction with several different data 
sources is arranged for wireless capture of encoded 
non-perceptible data directly from a light emitting de- 
vice, such as a television (101) of other video and/or 
audio output device. The data is part of the ordinary pro- 
gram displayed by the device (101 ) and yet indiscernible 
by a viewer or listener. The indiscemible data may be 
information needed to order those services or products, 
including prices, delivery intervals, shipping details, 
coupon offers, and so-on. After capture, the device in- 
cludes a memory arranged to store the data for later use, 
and communications circuitry (114 or 118) arranged to 
output at least portions of the data (or data derived from 
the captured data) to a remote agent or device 
(120.121), such as via a telephone call (via 140 or 109 
or 112) or a data transfer to a point of sale device. 
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Description 

Field ot the Invention 

This invention relates to a device and a method for 5 
the wireless capture of encoded non-perceptible data 
from a light emitting device such as a television or other 
video and/or audio output device, and subsequent use 
of the captured data to initiate a transaction, such as 
placing a telephone call and transmitting a message to 
a remote agent or automated attendant, or delivering the 
data to a point of sale agent directly or indirectly via a 
telecommunications connection, or to operate another 
device. 

Background of the Invention 

In a copending application entitled "System and 
Method of Capturing Encoded Data Transmitted Over a 
Communications Network in a Video System" filed on 
behalf of Isenberg and Tuomenoksa on March 18,1 994, 
Serial No. 08/21 0802 and assigned to the same assign- 
ee as the present application, it was recognized that 
non-perceptible information can be encoded in a televi- 
sion program, captured in a "set top box", and thereafter 
used to make telephone calls. The set top box is an elec- 
trical device that is "hard-wired* directly in the circuit be- 
tween the video services network and the television set 
(as a stand atone unit or alternatively as part of the tel- 
evision set itself) and thus, unfortunately, is not portable. 
Also, the set top box is dedicated to one particular tele- 
vision (or other video source) and thus cannot be shared 
among several sources. Finally, the user of the set top 
box may be uncertain as to the nature of the information 
captured therein. 

Summary of the Invention 

In accordance with our invention, a device which 
can be portable and therefore used iri conjunction with 
several different data sources is arranged for wireless 
capture of encoded non-perceptible data directly from a 
light emitting device, such as a television or other video 
and/or audio output device. The data is part of the ordi- 
nary program displayed by the device and yet indiscern- 
ible by a viewer or listener. As an example, a television 
monitor may be located in a kiosk or other public area, 
and display a program advertising selected services and 
products. The indiscernible data may be information 
needed to order those services or products, including 
prices, delivery intervals, shipping details, coupon of- 
fers, and so-on. 

Capture can be effected by decoding subliminally 
modulated video broadcast signals, in a manner sug- 
gested in U. S. Patent 4,607,031 , issued on February 
21, 1989. to R. S. Broughton, et aL, the teachings of 
which are incorporated herein by reference. Alternative- 
ly, a modulated carrier, which, for example, can be an 



amplitude shift keyed (ASK) or frequency shift keyed 
(FSK) signal having its primary frequency components 
at afrequency (e.g. 25 kHz) not usually present in a con- 
venttonal television signal, can be added only to the lu- 
minance component of a television signal, and the data 
can thereafter be captured using bandpass filtering and 
an ASK or FSK receiver. After capture, the device in- 
cludes a memory arranged to store the data for later use, 
and communications circuitry arranged to output at least 
portions of the data (or data derived from the captured 
data) to a remote agent or device, such as via a tele- 
phone call or a data transfer to a point of sale device. 

In accordance with the present invention, the en- 
coded non -perceptible data includes associated dialing, 
routing, or identification information, so that other por- 
tions of the captured data can be transmitted to an ap- 
propriate final destination or device. In this context, the 
final destination can be (a) a remote k)cation, such as a 
platfonm arranged to process and consummate transac- 
tions, or (b) a point of sale system or other device (e.g., 
TV, stereo, VCR Plus, smart mouse device, etc.). The 
other portions of the captured data that consfrtutes a 
payload to be communicated to the final destination or 
device, can, tor example, be product or coupon informa- 
tion that is related to the display from which the non- 
perceptible data was captured, or programming or con- 
trol data. 

Further in accordance with the present inventton, 
the capture device includes a built-in display, such as 
an LCD display, and the encoded non-perceptible data 
includes display information, so that at least portions of 
the captured data can be discerned by a user of the de- 
vice, and the user can thus be kept informed as to the 
operation of the device. The memory and the commu- 
nications circuitry in the capture device are advanta- 
geously arranged so that, in a transaction initiated by 
the devtee using captured infonmatton, a two-way dialog 
may be initiated between the user and an agent at a re- 
nrK>te tenminal or a POS system, and information may 
be transmitted to and stored in the device for later re- 
trieval. 

Brief Description of the Drawing 

The present invention will be more fully appreciated 
by consideration of the following Detailed Descriptton, 
which should be read in light of the accompanying draw- 
ing in which: 

Fig. 1 is a schematic drawing of one embodiment 

of a system arranged in accordance with the 

present invention for wireless capture of encoded 

data transmitted with a television signal; 

Fig. 2 is a block diagram illustrating the components 

of capture device 110 of Fig. 1; 

Fig. 3 is a diagram illustrating the format of captured 

data; and 

Fig. 4 is a flow diagram of the process by which data 
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embedded in a subliminal visual or non-audible 
message is captured and used. 

Detailed Description 

Referring first to Fig. 1 , there is shown a schematic 
drawing of one embodiment of the present invention ar- 
ranged for the wireless capture of encoded non-percep- 
tible data from a light emitting device such as television 
101 , and subsequent use of the captured data to initiate 
a transaction which might be handled by a person at an 
agent terminal 120. or a person at another terminal 121 
that is supported by an interactive service platform 130. 

A television signal is received by television 101 via 
over-the-air transmission, as from transmission tower 
102 via a transmission link 103, or alternatively is sup- 
plied by a cable TV connection, or from a VCR/video- 
tape. The video portion of the television signal is dis- 
played on the picture tube or other visual display area 
1 05 of a monitor, computer or a television 1 01 , while the 
audio portbn of the television signal is played through 
or by projected from speaker 106. Associated with the 
program that is seen and/or heard by a viewer/listener 
is non-discernible encoded data that is transmitted as 
part of the audio and/or video signal. For example, the 
program may be an advertisement for a service or prod- 
uct, and the indiscernible data may create a subliminal 
visual pattern on the visual display area, that when prop- 
erly received and decoded, contains infomnation need- 
ed to order those sewices or products, including prices, 
delivery intervals, shipping details, coupon offers, and 
so-on. 

Encoding, transmission and decoding of non-dis- 
cernible data may be accomplished as described in a 
patent application entitled "System and Method for En- 
coding Digital Information in a Television Signal" filed on 
behalf of T. Sizer and assigned to the same assignee as 
the present application, which co-pending application is 
incorporated herein by reference. Briefly, it was found 
using a spectrum analyzer on a typical video signal, that 
there are comparatively large frequency components at 
the line rate and at the frame rate and Its hamnonlcs, but 
that between these frequencies, there are other fre- 
quency bands in which little infomnation is carried. One 
such frequency band is between 1 5 and 30 kHz. By add- 
ing a low level carrier signal or tone at a frequency in 
this band, say 25kHz, the video image is not degraded, 
but a properly tuned decoder can receive and decode 
the encoded informatbn. In this way, digital information 
can be subliminally inserted in a video signal by adding 
to the video signal an amplitude shift keyed (ASK) or 
frequency shift keyed (FSK) carrier signal, and the dig- 
ital information can later be recovered using ASK or FSK 
decoding. Encoded data can also be inserted in a tele- 
vision signal and recovered by a receiver that responds 
to the picture displayed on the television, in the manner 
described in patent 4,807,031 cited above. 

The program displayed on television 101 is sensed 



by a portable capture device 110, which is described in 
more detail in connection with Fig. 2. When the non-dis- 
cernible encoded data is part of the video signal, such 
sensing takes the form of light from display area 105 
5 being collected by an optical lens 117, while when the 
non-discemible encoded data is part of the audio signal, 
such sensing takes the form of sound from speaker 1 06 
being collected by a microphone (211 in Fig. 2). In either 
event, there is a wireless communication of the informa- 
tion from television 101 to capture device 110 such that 
the encoded data is Viewed' or "heard" rather than be- 
ing transmitted through a wired connection, or through 
a more conventional radb frequency communication. 

In addition to receiving non-discernible encoded da- 
ta, capture device 110 has the capability of (a) storing 
the encoded data, (b) transmitting (or otherwise output- 
ting) the encoded data (or information retrieved using 
such data) to a remote location or device, and (c) dis- 
playing the encoded data to a user. The data may in- 
clude information for initiating a transactton or originat- 
ing a telephone call, as well as additional information (e. 
g., price and ordering information) associated with the 
television program being displayed on television 101. 
The remote location may be agent terminals 1 20 or 1 21 , 
interactive sen^ice platform 130, or a point of sale sys- 
tem, and the communication between the capture de- 
vice 110 and the remote location may be one-way or 
two-way. 

In the case where encoded data is to be output from 
capture device 1 10 via a telephone call, the call may be 
initiated and transacted in one of several ways, as illus- 
trated in Fig. 1. First, the data may be transmitted over 
a wireless link to an associated telephone base station 
109 that is, in turn, connected to a telephone line 111 
that is part of a telecommunications network 115. Com- 
munication between capture device 110 and base sta- 
tion 109 may be accomplished in a variety of ways; as 
an example, the same signaling and transmission can 
be used as is presently used in communication between 
a cordless telephone handset and its associated base 
station. Second, capture device 110 nnay be directly 
connected to a telephone line 112 through an interface 
that includes a standard telephone jack 114. Third, cap- 
ture device 110 may include the functranality of a cellular 
telephone, so that the data may be communicated to a 
remote location via a wireless call placed to a mobile 
telephone switching office (MTSO) 140, which is in turn 
connected to telecommunications network 115. 

As explained more fully in connection with Fig. 2, 
capture device 110 may include a dual tone multi-fre- 
quency (DTMF) transceiver arranged to generate a se- 
ries of audible tones. Accordingly, afourth way to initiate 
a telephone call would apply portbns of the data to the 
transmitter (tone generator) portion of the DTMF trans- 
ceiver. Capture device 110 nnay then be placed in juxta- 
position with the microphone portion of a conventional 
telephone handset, such that the audible tones gener- 
ated in capture device 110 are acoustically coupled 
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through the handset to receivers in teleconnmunications 
network 115. 

Capture device 110 can also output encoded data 
directly, without making a telephone call. This is useful 
in conjunction with a point of sale system or the like, 5 
which includes a docking station or other interface to 
which the capture device may be connected, or an I R 
receiving device or a device responding to DTMR 

If an outgoing telephone call is initiated by capture 
device 110, the call can illustratively be routed through 
telecommunications network 115 directly to an agent 
terminal 120, or to an interactive sen^ice platform 1 30 to 
which is connected another agent terminal 121. When 
a connection is completed to platform 1 30. the call may 
be routed by a switch matrix 131 to a voice response 
unit 132 that operates under the control of a processor 
1 34. Information included in the captured encoded data 
can be provided to the attendants, or processor 1 34 can 
perfomi a look-up and retrieval operation in database 
1 31 , using encoded data as a key, in order to effectuate 
a desired transaction. 

For certain transactions, information connected 
with a transaction may also be transmitted from service 
platform 1 30 to capture device 1 1 0, either for display to 
a user, storage for later use, or output to a point of sale 
system or device. This information may, for example, be 
confirmation information, indicating, that a transaction 
has been effected, or details regarding a just completed 
transaction, such as ticket information relating to a travel 
resenration. 

Referring now to Fig. 2, there is shown a block dia- 
gram illustrating in more detail the components in one 
embodiment of capture device 110 of Fig. 1. If the en- 
coded non-perceptible data is inserted in the video por- 
tion of a television signal, the images displayed on visual 
display area 105 are captured by optical lens 117 and 
an associated photodetector 203, which is arranged to 
supply an electrical signal to filter/receiver 215 repre- 
senting the image. On the other hand, if the encoded 
non-perceptible data is inserted in the audio portion of 
a televiston signal, the output from speaker 106 is cap- 
tured by a microphone 211 and an associated amplifier 
213, which is arranged to supply an electrical signal to 
filter/receiver 215 representing the sound energy. In ei- 
ther event, the encoded non-perceptible data Is decod- 
ed in filter/receiver 215 in a manner consistent with the 
manner in which the original data was encoded. Thus, 
filter/receiver 215 can be arranged to perform the same 
functionality as the elements illustrated in Fig. 4 of the 
above referenced Broughton et al patent, or, alterna- 
tively, the operation of filter/receiver 215 can be as de- 
scribed in conjunction with Figs. 4 or 7 in copending ap- 
plication of TSizer. In each instance, it is noted that cap- 
ture device 110 receives and recovers encoded data by 
"watching" or "listening to" the output of television 101 
(or another similar display device), and that the capture 
device 110 is not connected totelevisran 101 orthe other 
display device by a cable or wires. 



The output of fitter/receiver 215, representing the 
data captured by capture device 110, is applied to a mi- 
croprocessor 220, which is arranged to perform various 
data processing and control functions in coordination 
with programs stored in an associated memory 221 as 
well as inputs received from a user of capture device 
110. Generally speaking, capture device 110 may oper- 
ate in an active or a passive mode. In the active mode, 
capture device 110 responds only to user initiated cap- 
ture commands, in the passive mode, capture device 
110 captures all of the data sent to it while in that mode. 
In either mode, captured data may be routed by micro- 
processor to memory 221 and stored for use at a later 
time, as determined by the user. The captured data may 
also be displayed on a display 225, such as a liquid crys- 
tal display, so that a user will be aware of exactly what 
informatbn was captured. 

The control programs contained in memory 221 can 
implement numerous functions, which, broadly speak- 
ing, can be categorized as (a) f unctbns relating to data 
capture, such as SAVE and AUTOSAVE functions, (b) 
functions relating to output of already captured data, 
such as DIAL and SEND DATA functions, (c) functions 
relating to menrrary manipulation, such as the RECALL 
and DELETE functbns. The above enumerated func- 
tions are initiated when respective buttons 241 - 246 are 
activated, and the activation signal output from a button 
is applied to microprocessor 220 via a button interface 
240. Each of the functions is discussed below. 

When the SAVE function is initiated by activation of 
button 241, capture device 110 is in the active mode, 
and the encoded data contained in the current audio or 
video output of television 101 is captured. This is ac- 
complished, for example, by activating optical lens 117 
and photo detector 203, or microphone 211 and ampli- 
fier 213, for a relatively brief period of time, typically on 
the order of several seconds, so that encoded data as- 
sociated with the current program being displayed on 
display area 1 05 or played through the toudspeaker 1 06 
is processed by filter/receiver 21 5 and stored in memory 
221. A visual or auditory cue may be contained in the 
video or audio program, to indicate to a user of a capture 
device that there is data to be captured. When the SAVE 
function is activated, mbroprocessor 220 may be ar- 
ranged to control display 225 so as to indicate the cur- 
rent status, e.g. , that the capture device is in the process 
of receiving and capturing data. When the data has been 
captured, microprocessor 220 may be an-anged to con- 
trol display 225 so as to actually display some or all of 
the data, so that the user is now aware of information 
that had theretofor been subliminal and indiscemible. 
The display can also indicate that the device is 'aimed' 
correctly. 

When the AUTOSAVE function is initiated by acti- 
vation of button 244, capture device 110 is in the passive 
mode. In this mode, capture device 110 captures a se- 
ries of encoded data that may be contained in each of 
a series of programs that can be viewed/heard on tele- 
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vision 101. This is accomplisliecl, tor example, by acti- 
vating optical lens 117 and photo detector 203. or mi- 
crophone 211 and amplifier 213, tor a relatively long pe- 
riod of time, typically on the order of several minutes or 
hours. Encoded data captured from the various pro- s 
grams being displayed on display area 105 or played 
through budspeaker 1 06 over a long period of time are 
processed by filter/receiver 215 and stored in memory 
221. As with the SAVE function, when the AUTOSAVE 
function is activated, microprocessor 220 may be ar- 
ranged to control display 225 so as to indicate the cur- 
rent status, e.g. , that the capture device is in the process 
of receiving and capturing data. 

When a user of capture device 110 desires to use 
any ot the data stored in memory 221, including data 
stored a result of the AUTOSAVE function, the contents 
of memory 221 may be retrieved and displayed on dis- 
play 225 by initiating the RECALL function upon activat- 
ing button 245. This permits a user to bo aware of ex- 
actly what information was previously captured and 
stored. Through this function, the user will be able to 
selectively peruse or scroll through the stored data, so 
as to retain or use selected portions thereof. Activation 
of button 246 initiates the DELETE function, whereby 
portions of the data stored in memory 221 are selectively 
erased. In order to conserve and thereby "recycle" to 
storage capacity of memory 221, microprocessor 220 
may also be arranged to manage the contents in mem- 
ory 221, so that the oldest information stored therein 
could be cycled out and deleted as new information is 
captured and stored. 

Tuming now to consideration of the use of already 
captured data, it is an important tunctionatity of capture 
device 1 1 0 to permit such data to be output upon a com- 
mand from a user. Such data, when output, can be used 
to (a) make a telephone call, and, when the call is an- 
swered, to provide information to a live or automated 
attendant that receives the call, or (b) initiate a transac- 
tion, such as by interacting with a special purpose ter- 
minal or kiosk arranged to receive the stored data, or a 
device such as a VCR Plus or a stereo. In accordance 
with the arrangement shown In Fig. 2, data output via a 
telephone call can be initiated in several ways. 

First, captured data can be routed by microproces- 
sor 220 from memory 221 to a modem 246 and then to 
a base station interface 250, so that the information can 
be communicated to telephone base 109 of Fig. 1. This 
in turn allows a conventional telephone call to be origi- 
nated from telephone base 109. 

Second, captured data can be routed through mo- 
dem 246 to a wire interface 253, which allows capture 
device 110 to be physcally connected to a terminal or 
docking station for information off loading. 

Third, captured data can be routedtoa DTMFtrans- 
ceiver 247, which includes a tone generator arranged to 
output DTMF tones. These tones can be directly con- 
nected to a telephone line 11 2 via a standard connection 
such as RT11 jack 114. Alternatively, capture device 110 



can be placed in juxtaposition with the microphone in a 
telephone handset, and the audible tones can be ap- 
plied and used much like an audible frequency autodi- 
aler 

Fourth, captured data can be routed through mo- 
dem 246 to a cellular telephone 252, which can initiate 
a cellular call. When a cellular connection is completed, 
the captured data can be output in encoded form or as 
audible tones. 

For each of the arrangements just described, a tel- 
ephone call may be originated using the DIAL functbn. 
This function is initiated by activation of button 242. With 
this function, dialing or routing information, which is part 
of the data captured by capture device 110 and now 
stored in mennory 221, is retrieved. This infomriation is 
then further processed, so that a telephone call can be 
initiated. In some instances, a call is originated by ap- 
plicatbn ot stored data to DTMF transceiver 247, so that 
audible tones can be output. In other instances, a data 
call can be initiated by appropriately formatting stored 
data. In yet other instances, such as when a cellular call 
is dialed, stored dialing infomnation nnay be applied to 
an outgoing number register The exact manner in which 
informatton is output from capture device 110 can vary, 
in accordance with several embodiments ot the present 
invention. 

While the DIAL function is used to output certain 
captured information needed to initiate a telephone call 
(or other data output function), the SEND DATA function, 
initiated by activation of button 243, is used to output 
other portions of the captured data stored in memory 
221, pertaining, for example, to a transaction that the 
user desires to make. This additional data is retrieved 
from memory 221 and appropriately transmitted, de- 
pending upon the rmnner and type of telephone call in- 
volved. Thus, data may be output either via base station 
interface 250 to telephone base 109, or via RJ 11 jack 
114 onto telephone line 112, or via cellular phone 252 
to MTS0 1 40. Note that if output occurs directly, without 
a telephone call, the SEND DATA function may cause 
informatton from memory 221 to be output via wire In- 
terface 253. During the SEND DATA function, the infor- 
mation being output nnay also be displayed on display 
225, This allows a person using the capture device to 
identify the information that was captured, and use the 
information in an actual transaction. 

It Is to be noted here that capture device 110 of Fig. 
2 may optionally include an automatic speech recogni- 
tion circuit 245, an infrared (I R) transmitter 235. a hand- 
set 230 and a keypad 247. The purpose of 1 R transmitter 
235 is to allow captured data to be output directly to a 
device such as a point of sale system or a VCR or the 
like, via infrared transmission. The purpose ot speech 
recognition circu it 245 is to convert captured data to spo- 
ken phrases, which can be played to a user or to an at- 
tendant after a telephone call is initiated. The purpose 
of handset 230 and keypad 247 is to altow capture do- 
vbe 110, in additbn to being used in accordance with 
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the present invention, to be used to complete conven- 
tional dialed telephone calls. 

In one application of the present invention, a tele- 
vised advertisement tor a product includes additional or- 
dering or "coupon" information embedded in a sublimi- 
nal visual pattern within the video image displayed on 
visual display area 105. The coupon information de- 
scribes the item(s) offered for sale, the regular price, the 
coupon value, and contains additbnal information need- 
ed for ordering. This embedded inf omriat'ion is transmit- 
ted between television 101 and capture device 110 at a 
rate such as 60 bits per second, that is sufficient such 
that the necessary information can be transmitted in a 
very brief period of time. In the application just de- 
scribed, the embedded product information, dialing 
number, coupon infornnation and necessary checkbrts 
together comprise approximately 2,000 bits of informa- 
tion, so that between three and four seconds would be 
required for transmission. Once the coupon information 
is stored in capture device 110, the user of the capture 
device can supplement the coupon infornnation with 
user entered data which can be entered using keypad 
247. This supplemental information may include credit 
card or other billing infornnation pertaining to the user, 
as an example. 

As shown in Fig. 3, the information represented by 
the encoded data can be organized in several fields. Us- 
ing the coupon application as an example, a first field 
301 contains a telephone number which may be used 
to originate a telephone call in order to order the product 
to which the coupon applies. Illustratively, this will be a 
toll-free or "800" number Other infonmatbn associated 
with the coupon can include a product description in field 
303, an expiration date in field 305, the time that the 
advertisement was televised in field 307, the number of 
the television channel running the ad in field 309, and a 
serial number for the ad in field 311 , identifying either or 
both the particular ad and its sequence in an ad cam- 
paign. It is to be noted that varbus other information can 
also be included in the information illustrated in Fig. 3, 
such as keywords or attributes describing the received 
encoded data, as well as information indicating that a 
product is available in particular coIch-s or sizes. The key- 
words would be useful in retrieving from memory 221, 
information relating to specific products or advertisers. 

The process by which information embedded in a 
subliminal visual or audible message is captured and 
used is illustrated in flow diagram form in Fig. 4. As in- 
dicated previously, two general modes of operation are 
contemplated, namely a "passive" nnode and an "active" 
mode, both of which folbw a "start" step 401. 

In the passive mode, a user desires to operate cap- 
ture device 110 in an unattended manner, and to capture 
all of the encoded data that nnay be subliminally dis- 
played on a display device 105 or output from speaker 
1 06 over a relatively long period of time, typically several 
minutes or hours, until the user, at some later time, de- 
sires to discontinue or terminate the capture. This mode 



is initiated using AUTOSAVE functionality, by activating 
button 244. When this is done, a positive response is 
obtained in step 502, so that data capture, storage and 
display, all as described below in connection with step 

5 507, occur. While in the passive mode, capture is termi- 
nated by subsequent activation of the same button 224. 

In the active mode, a user operates capture device 
110 in real time, typically while viewing a video display 
or listening to an audio presentation. During the display 

10 or presentation, the user sees or hears a corresponding 
televised advertisement or other indication that imper- 
ceptible information Is available for capture. The user 
may, at this time, be instructed to activate SAVE button 
241, or the user may know, from prevraus experience, 

1B how to use the device. When button 241 is activated, a 
positive result occurs in step 503, thereby capturing the 
imperceptible encoded data, displaying all or a portion 
of the data on display 225, and storing same in memory 
221 , all in step 507. Until the SAVE button Is activated, 

20 capture device 110 may be in a "wait state" 505. 

The user of capture device 110 determines that the 
appropriate data has been captured by reviewing the in- 
fornnation seen in display 225. After the captured data 
has been reviewed and determined to be correct, acti- 
os vation of the DIAL button 242 on the portable capture 
device 110 in step 509 initiates step 51 3, by which stored 
dialing and/or routing information, illustratively in field 
401 of Fig. 4, is recalled from memory 221 . A call is then 
originated launched, such by transmitting the dialing in- 

30 formation from memory 221 through base station inter- 
face 250 to base station 109 and thence to telephone 
line 111 and telecommunications network 115 to inter- 
active service platform 130. The call may be a local or 
long distance call, made using a "plain old telephone 

3S service" (POTS) line; alternatively, a toll-free call may 
be launched using an 800 number Desirably, automatic 
number identification (ANI), caller ID, and other features 
available in the telecommunications network may be 
used to provide additbnal information to the called party, 

40 in additbn to the captured data that is transmitted in sue- . , 
ceeding steps in the process. Until the DIAL button is 
activated, capture device 110 may be in a "wait state" 
511. 

After the call has been originated, a determination 
45 is made in step 515 that the call has been answered and 
that the SEND DATA button 243 has been pushed. (Until 
the SEND DATA button is activated, capture device 110 
may be in a "wait state" 517.) When a positive result 
occurs in step 515, information contained in fields 
so 403-411 is transmitted to the called destination in step 
519. As stated previously, from the point of view of the 
called party, the information received includes details 
concerning the advertised product, the identity of the 
caller, and other Information associated with the item 
55 that was initially displayed or mentioned on the televi- 
sbn program. Advantageously, the advertiser will know 
nnore about their customer than ever before. This is be- 
cause the advertiser will be able to determine from the 
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transmitted information wliicli advertisement drew the 
business, how long it took the caller to respond to the 
ad. and the geographic location of the callers, based 
upon ANI or caller ID. It will be easy to tell which ads in 
a campaign are most successful. 

In accordance with one optional arrangement of the 
present invention, when data Is captured and stored in 
step 507. the newly collected data is compared with pre- 
viously stored data. When a match is found, any dupli- 
cate data is dropped. In addition, the informatbn in var- 
ious fields is examined, to assure that the data captured 
is in the appropriate format. If an error is detected in any 
data, the information may be deleted, and "new" data 
collected. This is accomplished by repeating capture 
step 507 several times. 

In another application of the present invention, an 
interactive television capability is provided in connection 
with a game show or television drama. For a game show, 
an audience would be asked to vote on various aspects 
of the game. This could include questions such as which 
person should go on a date with the contestant, or which 
singer is the best? The audience would be instructed to 
press the "SAVE" button on their capture device at the 
appropriate time specified for their choice. The viewers 
vote would then be cast by depressing the DIAL button, 
communicating with the game show host, and then 
transmitting information indicating the vote by depress- 
ing the SEND DATA button. Alternately, one number 
could be transmitted and an automated interactive voice 
platform such as the InfoWorx™ platform available from 
AT&T could offer options to the caller. These calls could 
be toll free calls, or, alternatively, billed to the caller. 

In yet another application of the present invention, 
a video on demand capability can be provided. A viewer 
sees a video he or she would like to see advertised on 
a previewing network. An Instruction to press the DIAL 
button on the capture device is made on the previewing 
channel to indicate to the viewer that he or she can press 
the button to order a particular video. The call woukJ be 
placed to a voice response system used to reserve the 
video on demand. Information about which video the 
caller wanted to see is sent over the phone lines once 
the call is connected. The caller woukl merely have to 
acknowledge the order and select a time slot. 

The present invention can also be employed In the 
context of programming of a VCR, particularly where the 
user already has the capabilities provided by VCR Plus, 
a product that is commercially available. In this embod- 
iment, a code sequence associated with and represent- 
ing a particular television show constitutes the informa- 
tion included in a televised advertisement. When the ad 
is viewed, the viewer can press the SAVE button and 
send the sequences to the VCR Plus using IR transmit- 
ter 235. 

While the portability of capture devrce 11 0 is advan- 
tageous, it is to be noted that in some arrangements, 
the functions of capture device 110 and telephone base 
109 may be combined in a single stationary unit, which 



might resemble a cordless telephone base station. In 
this event, the functionality of capture device 110 would 
be included in the stationary unit, and that unit would be 
arranged, in the manner previously described, to collect 

s informatbn from the visual display area 1 05 and to pro- 
vide storage of the collected information. In this arrange- 
ment, the stationary unit could also be adapted to utilize 
input devices such as optical character recognition 
(OCR) or scanners that would provide an additional 

10 method of inputting telephone numbers and coupon 
and/or product information. The hand piece of this sta- 
tionary unit would be used to place a call and send the 
data. 

The present invention is not limited to live televised 

15 programs, but can also be operated in conjunction with 
stored video programs played from a VCR. In this ar- 
rangement, advertisements and other encoded informa- 
tion is included in videos at the time they are recorded 
or at the time they are mastered and put onto video cas- 

20 settes. A viewer watching a video with such an ad can 
respond immediately to resen/e another video, pur- 
chase one of the items advertised, or order catalogs au- 
tomatically. When a program containing subliminally en- 
coded data is viewed, the viewer simply presses the 

25 DIAL button, launching a telephone call, and then press- 
es the SEND DATA button, sending captured data, such 
as product information, to an agent, who can verify the 
order using the additional infomiation that was captured 
or stored in the device. 

30 The portability of the present invention can be used 
to advantage in implementing what can, in effect, be 
called a "Roaming Bulletin Board". A user can be attend- 
ing a conference or traveling through the airport where 
kiosks or bulletin boards are set up. In the case of a con- 

3B ference, an attendee would walkthrough the conference 
viewing the displays. When the attendee sees a display 
of interest, he or she points the capture device towards 
the display, and captures the non-discemible encoded 
data from either the displayed image or the sound out- 

40 put. Later, the user launches a telephone call to the tel: 
ephone number specified and Included in the captured 
data, and sends out coupon information that is also part 
of the captured data. 

It Is further to be noted that the present invention 

45 can be used in the context of automobiles and mobile 
telephones. When a capture device in accordance with 
the invention is placed in a car, it can receive and cap- 
ture non-discernible encoded data contained in the au- 
db output of the car radio. The captured data can in- 

50 elude a toll free or conventional telephone number for 
originating a mobile cellular telephone call from the car. 
The call may be placed to an interactive service plat- 
fomn, and additional captured data, such as coupon or 
similar information, can then be used to effectuate a 

55 transaction. 

Various modifications and adaptations of the 
present inventbn will be apparent to those skilled in the 
art. For example, the functions provided by the present 
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inventbn may be combined with those used in a per- 
sonal digital assistant (PDA) or a laptop computer. If de- 
sired, a simultaneous voice and data (SVD) modem of 
the type available now from AT&T could be provided in 
capture device 110 to allow communication both by 
voice and data transfer, at the same time. Varbus func- 
tions can also be added to the processes and applica- 
tions described. For example, telephone base 109 may 
be programmed to dial a number or series of numbers 
at specific times, when certain amounts of information 
or data have been collected, or based upon some crite- 
ria. Capture device 110 could then send collected infor- 
mation "in bulk" to interactive service platform 130. For 
this purpose, modem 246 could be used, so as to send 
the information more efficiently. 



Claims 

1. . Apparatus for initiating a transaction, comprising 

means arranged for wireless capture of data 
directly from an audio/visual display, said data 
being Indlscemible by a viewer or listener, and 
means arranged to transfer at least portions of 
said data to a separate transaction processor 
to initiate said transaction. 

2. The invention defined in claim 1 wherein said data 
includes dialing infomnation for originating a tele- 
phone call, and wherein said transfer means 
includes means for initiating said telephone call 
using said dialing infonnation. 

3. The invention defined in claim 1 wherein said appa- 
ratus further includes 

means responsive to said capture means for 
storing said data, and 

wherein said transfer means includes means 
for coupling information from said storing 
means directly to a separate transaction proc- 
essor. 

4. The invention defined in claim 1 wherein said cap- 
ture means is in a portable capture device and said 
transfer means is in a base station, and wherein 
said apparatus further includes means for transfer- 
ring information from said portable capture device 
to said base station. 

5. The invention defined in claim 1 wherein said 
audio/visual display includes non-perceptible 
encoded data, and wherein said capture means is 
arranged to receive and decode said encoded data. 

6. The Invention defined in claim 5 wherein said non- 
perceptible encoded data includes an amplitude 



shift keyed (ASK) or frequency shift keyed (FSK) 
carrier signal added to a television signal, and 
wherein capture is effected by detecting and decod- 
ing said carrier signal. 

5 

7. A system for capturing data comprising 

means for wireless capture of indiscernible 
data embedded in an audioA/ideo display; and 
10 means for outputting at least portions of said 

captured indiscernible data to a remote device 
via a telephone call. 

8. Apparatus for initiating a transaction relating to a 
15 program appearing on a television display, said 

apparatus comprising 

means arranged for wireless capture of data 
directly from said televlsbn display, said data 
20 being indiscernible to a viewer of said television 

display; 

means responsive to said captured data for 
obtaining a telephone number, said telephone 
number being associated with a remote loca- 
25 tion arranged to perform transactions associ- 

ated with said program appearing on said tele- 
visbn display; 

means for initiating a telephone call using said 
telephone number; and 
30 means for transmitting to said remote locatbn. 

during said telephone call, additional informa- 
tion contained in said captured data, said addi- 
tional information also pertaining to said pro- 
gram. 

3$ 

9. A system for responding to a televised advertise- 
ment, comprising 

means for embedding non-discernible infonna- 
40 tion in said advertisement, said information 

including (a) a telephone number for a rernoxe 
processor arranged to process orders for the 
items displayed in said advertisement, and (b) 
other informatKin relating to said items dis- 
45 played in said advertisement; 

means arranged for wireless capture of said 
non-discernible informatbn directly from said 
televised advertisement; and 
means responsive to said captured information 
so for (a) placing a telephone call using said tele- 

phone number and (b) transmitting said other 
infonnation to said remote processor via said 
telephone call. 

ss 10, A method for initiating a transactbn, comprising the 
steps of 

capturing data directly from an audio/visual dis- 
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play, said data being Indiscernible by a viewer 
or listener, and 

transferring at least portbns o1 said data to a 
separate transaction processor to initiate said 
transaction. ^ 

11. The method defined in claim 10 wherein said data 
includes dialing informatran for originating a tele- 
phone call, and wherein said transferring step 
includes Initiating said telephone call using said 
dialing information. 

12. A method for capturing data comprising 

capturing, using wireless capture, indiscernible >5 
data embedded in an audio/video display; and 
outputting at least portions of said captured 
indiscernible data to a remote device via a tel- 
ephone call. 

20 

13. A method for initiating a transaction relating to a pro- 
gram appearing on a television display, said method 
comprising steps of 

using wireless capture data directly from said 25 
television display, said data being indiscernible 
to a viewer of said television display; 
responsive to said captured data, obtaining a 
telephone number, said telephone number 
being associated with a remote location 
arranged to perform transactions associated 
with said program appearing on said television 
display; 

initiating a telephone call using said telephone 
number; and ' 
transmitting to said remote location, during said 
telephone call, additional Information con- 
tained in said captured data, said additional 
information also pertaining to said program. 

40 

14. A method tor responding to a televised advertise- 
ment, comprising the steps of 

embedding non-discernible Information in said 
advertisement, said information Including (a) a 45 
telephone number for a remote processor 
arranged to process orders for the items dis- 
played in said advertisement and (b) other 
Infornnation relating to said items displayed In 
said advertisement; ^ 
capturing, using wireless capture, said non-dis- 
cernible information directly from said televised 
advertisement; and 

responsive to said captured information, (a) 
placing a telephone call using said telephone 
number and (b) transmitting said other informa- 
tbn to said remote processor via said tele- 
phone call. 
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